Exact Bayesian inference via data augmentation

Data augmentation is a common tool in Bayesian statistics, especially in the application of MCMC. Data augmentation is used where direct computation of the posterior density, π(θ |x), of the parameters θ , given the observed data x, is not possible. We show that for a range of problems, it is possible to augment the data by y, such that, π(θ |x,y) is known, and π(y|x) can easily be computed. In particular, π(y|x) is obtained by collapsing π(y, θ |x) through integrating out θ . This allows the exact computation of π(θ |x) as a mixture distribution without recourse to approximating methods such as MCMC. Useful byproducts of the exact posterior distribution are the marginal likelihood of the model and the exact predictive distribution.